<!-- HTML header for doxygen 1.8.6-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<title>OpenCV: Camera Calibration</title>
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../jquery.js"></script>
<script type="text/javascript" src="../../dynsections.js"></script>
<script type="text/javascript" src="../../tutorial-utils.js"></script>
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../search/searchdata.js"></script>
<script type="text/javascript" src="../../search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
//<![CDATA[
MathJax.Hub.Config(
{
  TeX: {
      Macros: {
          matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
          fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
          forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
          forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
          vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
          vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
          cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
          distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
          distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
          hdotsfor: ["\\dots", 1],
          mathbbm: ["\\mathbb{#1}", 1],
          bordermatrix: ["\\matrix{#1}", 1]
      }
  }
}
);
//]]>
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<!--#include virtual="/google-search.html"-->
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
  <td style="padding-left: 0.5em;">
   <div id="projectname">OpenCV
   &#160;<span id="projectnumber">4.5.2</span>
   </div>
   <div id="projectbrief">Open Source Computer Vision</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
</script>
<script type="text/javascript" src="../../menudata.js"></script>
<script type="text/javascript" src="../../menu.js"></script>
<script type="text/javascript">
$(function() {
  initMenu('../../',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="../../d6/d00/tutorial_py_root.html">OpenCV-Python Tutorials</a></li><li class="navelem"><a class="el" href="../../d9/db7/tutorial_py_table_of_contents_calib3d.html">Camera Calibration and 3D Reconstruction</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">Camera Calibration </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><h2>Goal </h2>
<p>In this section, we will learn about</p>
<ul>
<li>types of distortion caused by cameras</li>
<li>how to find the intrinsic and extrinsic properties of a camera</li>
<li>how to undistort images based off these properties</li>
</ul>
<h2>Basics </h2>
<p>Some pinhole cameras introduce significant distortion to images. Two major kinds of distortion are radial distortion and tangential distortion.</p>
<p>Radial distortion causes straight lines to appear curved. Radial distortion becomes larger the farther points are from the center of the image. For example, one image is shown below in which two edges of a chess board are marked with red lines. But, you can see that the border of the chess board is not a straight line and doesn't match with the red line. All the expected straight lines are bulged out. Visit <a href="http://en.wikipedia.org/wiki/Distortion_%28optics%29">Distortion (optics)</a> for more details.</p>
<div class="image">
<img src="../../calib_radial.jpg" alt="calib_radial.jpg"/>
<div class="caption">
image</div></div>
<p> Radial distortion can be represented as follows:</p>
<p class="formulaDsp">
\[x_{distorted} = x( 1 + k_1 r^2 + k_2 r^4 + k_3 r^6) \\ y_{distorted} = y( 1 + k_1 r^2 + k_2 r^4 + k_3 r^6)\]
</p>
<p>Similarly, tangential distortion occurs because the image-taking lense is not aligned perfectly parallel to the imaging plane. So, some areas in the image may look nearer than expected. The amount of tangential distortion can be represented as below:</p>
<p class="formulaDsp">
\[x_{distorted} = x + [ 2p_1xy + p_2(r^2+2x^2)] \\ y_{distorted} = y + [ p_1(r^2+ 2y^2)+ 2p_2xy]\]
</p>
<p>In short, we need to find five parameters, known as distortion coefficients given by:</p>
<p class="formulaDsp">
\[Distortion \; coefficients=(k_1 \hspace{10pt} k_2 \hspace{10pt} p_1 \hspace{10pt} p_2 \hspace{10pt} k_3)\]
</p>
<p>In addition to this, we need to some other information, like the intrinsic and extrinsic parameters of the camera. Intrinsic parameters are specific to a camera. They include information like focal length ( \(f_x,f_y\)) and optical centers ( \(c_x, c_y\)). The focal length and optical centers can be used to create a camera matrix, which can be used to remove distortion due to the lenses of a specific camera. The camera matrix is unique to a specific camera, so once calculated, it can be reused on other images taken by the same camera. It is expressed as a 3x3 matrix:</p>
<p class="formulaDsp">
\[camera \; matrix = \left [ \begin{matrix} f_x &amp; 0 &amp; c_x \\ 0 &amp; f_y &amp; c_y \\ 0 &amp; 0 &amp; 1 \end{matrix} \right ]\]
</p>
<p>Extrinsic parameters corresponds to rotation and translation vectors which translates a coordinates of a 3D point to a coordinate system.</p>
<p>For stereo applications, these distortions need to be corrected first. To find these parameters, we must provide some sample images of a well defined pattern (e.g. a chess board). We find some specific points of which we already know the relative positions (e.g. square corners in the chess board). We know the coordinates of these points in real world space and we know the coordinates in the image, so we can solve for the distortion coefficients. For better results, we need at least 10 test patterns.</p>
<h2>Code </h2>
<p>As mentioned above, we need at least 10 test patterns for camera calibration. OpenCV comes with some images of a chess board (see samples/data/left01.jpg &ndash; left14.jpg), so we will utilize these. Consider an image of a chess board. The important input data needed for calibration of the camera is the set of 3D real world points and the corresponding 2D coordinates of these points in the image. 2D image points are OK which we can easily find from the image. (These image points are locations where two black squares touch each other in chess boards)</p>
<p>What about the 3D points from real world space? Those images are taken from a static camera and chess boards are placed at different locations and orientations. So we need to know \((X,Y,Z)\) values. But for simplicity, we can say chess board was kept stationary at XY plane, (so Z=0 always) and camera was moved accordingly. This consideration helps us to find only X,Y values. Now for X,Y values, we can simply pass the points as (0,0), (1,0), (2,0), ... which denotes the location of points. In this case, the results we get will be in the scale of size of chess board square. But if we know the square size, (say 30 mm), we can pass the values as (0,0), (30,0), (60,0), ... . Thus, we get the results in mm. (In this case, we don't know square size since we didn't take those images, so we pass in terms of square size).</p>
<p>3D points are called <b>object points</b> and 2D image points are called <b>image points.</b></p>
<h3>Setup</h3>
<p>So to find pattern in chess board, we can use the function, <b><a class="el" href="../../d9/d0c/group__calib3d.html#ga93efa9b0aa890de240ca32b11253dd4a" title="Finds the positions of internal corners of the chessboard. ">cv.findChessboardCorners()</a></b>. We also need to pass what kind of pattern we are looking for, like 8x8 grid, 5x5 grid etc. In this example, we use 7x6 grid. (Normally a chess board has 8x8 squares and 7x7 internal corners). It returns the corner points and retval which will be True if pattern is obtained. These corners will be placed in an order (from left-to-right, top-to-bottom)</p>
<dl class="section note"><dt>Note</dt><dd>This function may not be able to find the required pattern in all the images. So, one good option is to write the code such that, it starts the camera and check each frame for required pattern. Once the pattern is obtained, find the corners and store it in a list. Also, provide some interval before reading next frame so that we can adjust our chess board in different direction. Continue this process until the required number of good patterns are obtained. Even in the example provided here, we are not sure how many images out of the 14 given are good. Thus, we must read all the images and take only the good ones.</dd>
<dd>
Instead of chess board, we can alternatively use a circular grid. In this case, we must use the function <b><a class="el" href="../../d9/d0c/group__calib3d.html#ga7f02cd21c8352142890190227628fa80" title="Finds centers in the grid of circles. ">cv.findCirclesGrid()</a></b> to find the pattern. Fewer images are sufficient to perform camera calibration using a circular grid.</dd></dl>
<p>Once we find the corners, we can increase their accuracy using <b><a class="el" href="../../dd/d1a/group__imgproc__feature.html#ga354e0d7c86d0d9da75de9b9701a9a87e" title="Refines the corner locations. ">cv.cornerSubPix()</a></b>. We can also draw the pattern using <b><a class="el" href="../../d9/d0c/group__calib3d.html#ga6a10b0bb120c4907e5eabbcd22319022" title="Renders the detected chessboard corners. ">cv.drawChessboardCorners()</a></b>. All these steps are included in below code:</p>
<div class="fragment"><div class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</div><div class="line"><span class="keyword">import</span> cv2 <span class="keyword">as</span> cv</div><div class="line"><span class="keyword">import</span> glob</div><div class="line"></div><div class="line"><span class="comment"># termination criteria</span></div><div class="line">criteria = (cv.TERM_CRITERIA_EPS + cv.TERM_CRITERIA_MAX_ITER, 30, 0.001)</div><div class="line"></div><div class="line"><span class="comment"># prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)</span></div><div class="line">objp = np.zeros((6*7,3), np.float32)</div><div class="line">objp[:,:2] = np.mgrid[0:7,0:6].T.reshape(-1,2)</div><div class="line"></div><div class="line"><span class="comment"># Arrays to store object points and image points from all the images.</span></div><div class="line">objpoints = [] <span class="comment"># 3d point in real world space</span></div><div class="line">imgpoints = [] <span class="comment"># 2d points in image plane.</span></div><div class="line"></div><div class="line">images = glob.glob(<span class="stringliteral">&#39;*.jpg&#39;</span>)</div><div class="line"></div><div class="line"><span class="keywordflow">for</span> fname <span class="keywordflow">in</span> images:</div><div class="line">    img = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv.imread</a>(fname)</div><div class="line">    gray = <a class="code" href="../../d8/d01/group__imgproc__color__conversions.html#ga397ae87e1288a81d2363b61574eb8cab">cv.cvtColor</a>(img, cv.COLOR_BGR2GRAY)</div><div class="line"></div><div class="line">    <span class="comment"># Find the chess board corners</span></div><div class="line">    ret, corners = <a class="code" href="../../d9/d0c/group__calib3d.html#ga93efa9b0aa890de240ca32b11253dd4a">cv.findChessboardCorners</a>(gray, (7,6), <span class="keywordtype">None</span>)</div><div class="line"></div><div class="line">    <span class="comment"># If found, add object points, image points (after refining them)</span></div><div class="line">    <span class="keywordflow">if</span> ret == <span class="keyword">True</span>:</div><div class="line">        objpoints.append(objp)</div><div class="line"></div><div class="line">        corners2 = <a class="code" href="../../dd/d1a/group__imgproc__feature.html#ga354e0d7c86d0d9da75de9b9701a9a87e">cv.cornerSubPix</a>(gray,corners, (11,11), (-1,-1), criteria)</div><div class="line">        imgpoints.append(corners)</div><div class="line"></div><div class="line">        <span class="comment"># Draw and display the corners</span></div><div class="line">        <a class="code" href="../../d9/d0c/group__calib3d.html#ga6a10b0bb120c4907e5eabbcd22319022">cv.drawChessboardCorners</a>(img, (7,6), corners2, ret)</div><div class="line">        <a class="code" href="../../df/d24/group__highgui__opengl.html#gaae7e90aa3415c68dba22a5ff2cefc25d">cv.imshow</a>(<span class="stringliteral">&#39;img&#39;</span>, img)</div><div class="line">        <a class="code" href="../../d7/dfc/group__highgui.html#ga5628525ad33f52eab17feebcfba38bd7">cv.waitKey</a>(500)</div><div class="line"></div><div class="line"><a class="code" href="../../d7/dfc/group__highgui.html#ga6b7fc1c1a8960438156912027b38f481">cv.destroyAllWindows</a>()</div></div><!-- fragment --><p> One image with pattern drawn on it is shown below:</p>
<div class="image">
<img src="../../calib_pattern.jpg" alt="calib_pattern.jpg"/>
<div class="caption">
image</div></div>
 <h3>Calibration</h3>
<p>Now that we have our object points and image points, we are ready to go for calibration. We can use the function, <b><a class="el" href="../../d9/d0c/group__calib3d.html#ga3207604e4b1a1758aa66acb6ed5aa65d" title="Finds the camera intrinsic and extrinsic parameters from several views of a calibration pattern...">cv.calibrateCamera()</a></b> which returns the camera matrix, distortion coefficients, rotation and translation vectors etc. </p><div class="fragment"><div class="line">ret, mtx, dist, rvecs, tvecs = <a class="code" href="../../d9/d0c/group__calib3d.html#ga687a1ab946686f0d85ae0363b5af1d7b">cv.calibrateCamera</a>(objpoints, imgpoints, gray.shape[::-1], <span class="keywordtype">None</span>, <span class="keywordtype">None</span>)</div></div><!-- fragment --><h3>Undistortion</h3>
<p>Now, we can take an image and undistort it. OpenCV comes with two methods for doing this. However first, we can refine the camera matrix based on a free scaling parameter using <b><a class="el" href="../../d9/d0c/group__calib3d.html#ga7a6c4e032c97f03ba747966e6ad862b1" title="Returns the new camera intrinsic matrix based on the free scaling parameter. ">cv.getOptimalNewCameraMatrix()</a></b>. If the scaling parameter alpha=0, it returns undistorted image with minimum unwanted pixels. So it may even remove some pixels at image corners. If alpha=1, all pixels are retained with some extra black images. This function also returns an image ROI which can be used to crop the result.</p>
<p>So, we take a new image (left12.jpg in this case. That is the first image in this chapter) </p><div class="fragment"><div class="line">img = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv.imread</a>(<span class="stringliteral">&#39;left12.jpg&#39;</span>)</div><div class="line">h,  w = img.shape[:2]</div><div class="line">newcameramtx, roi = <a class="code" href="../../d9/d0c/group__calib3d.html#ga7a6c4e032c97f03ba747966e6ad862b1">cv.getOptimalNewCameraMatrix</a>(mtx, dist, (w,h), 1, (w,h))</div></div><!-- fragment --> <h4>1. Using <b><a class="el" href="../../d9/d0c/group__calib3d.html#ga69f2545a8b62a6b0fc2ee060dc30559d" title="Transforms an image to compensate for lens distortion. ">cv.undistort()</a></b></h4>
<p>This is the easiest way. Just call the function and use ROI obtained above to crop the result. </p><div class="fragment"><div class="line"><span class="comment"># undistort</span></div><div class="line">dst = <a class="code" href="../../d9/d0c/group__calib3d.html#ga69f2545a8b62a6b0fc2ee060dc30559d">cv.undistort</a>(img, mtx, dist, <span class="keywordtype">None</span>, newcameramtx)</div><div class="line"></div><div class="line"><span class="comment"># crop the image</span></div><div class="line">x, y, w, h = roi</div><div class="line">dst = dst[y:y+h, x:x+w]</div><div class="line"><a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">cv.imwrite</a>(<span class="stringliteral">&#39;calibresult.png&#39;</span>, dst)</div></div><!-- fragment --> <h4>2. Using <b>remapping</b></h4>
<p>This way is a little bit more difficult. First, find a mapping function from the distorted image to the undistorted image. Then use the remap function. </p><div class="fragment"><div class="line"><span class="comment"># undistort</span></div><div class="line">mapx, mapy = <a class="code" href="../../d9/d0c/group__calib3d.html#ga7dfb72c9cf9780a347fbe3d1c47e5d5a">cv.initUndistortRectifyMap</a>(mtx, dist, <span class="keywordtype">None</span>, newcameramtx, (w,h), 5)</div><div class="line">dst = <a class="code" href="../../da/d54/group__imgproc__transform.html#gab75ef31ce5cdfb5c44b6da5f3b908ea4">cv.remap</a>(img, mapx, mapy, cv.INTER_LINEAR)</div><div class="line"></div><div class="line"><span class="comment"># crop the image</span></div><div class="line">x, y, w, h = roi</div><div class="line">dst = dst[y:y+h, x:x+w]</div><div class="line"><a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">cv.imwrite</a>(<span class="stringliteral">&#39;calibresult.png&#39;</span>, dst)</div></div><!-- fragment --><p> Still, both the methods give the same result. See the result below:</p>
<div class="image">
<img src="../../calib_result.jpg" alt="calib_result.jpg"/>
<div class="caption">
image</div></div>
<p> You can see in the result that all the edges are straight.</p>
<p>Now you can store the camera matrix and distortion coefficients using write functions in NumPy (np.savez, np.savetxt etc) for future uses.</p>
<h2>Re-projection Error </h2>
<p>Re-projection error gives a good estimation of just how exact the found parameters are. The closer the re-projection error is to zero, the more accurate the parameters we found are. Given the intrinsic, distortion, rotation and translation matrices, we must first transform the object point to image point using <b><a class="el" href="../../d9/d0c/group__calib3d.html#ga1019495a2c8d1743ed5cc23fa0daff8c" title="Projects 3D points to an image plane. ">cv.projectPoints()</a></b>. Then, we can calculate the absolute norm between what we got with our transformation and the corner finding algorithm. To find the average error, we calculate the arithmetical mean of the errors calculated for all the calibration images. </p><div class="fragment"><div class="line">mean_error = 0</div><div class="line"><span class="keywordflow">for</span> i <span class="keywordflow">in</span> range(len(objpoints)):</div><div class="line">    imgpoints2, _ = <a class="code" href="../../d9/d0c/group__calib3d.html#ga1019495a2c8d1743ed5cc23fa0daff8c">cv.projectPoints</a>(objpoints[i], rvecs[i], tvecs[i], mtx, dist)</div><div class="line">    error = <a class="code" href="../../d2/de8/group__core__array.html#ga7c331fb8dd951707e184ef4e3f21dd33">cv.norm</a>(imgpoints[i], imgpoints2, cv.NORM_L2)/len(imgpoints2)</div><div class="line">    mean_error += error</div><div class="line"></div><div class="line"><a class="code" href="../../df/d57/namespacecv_1_1dnn.html#a701210a0203f2786cbfd04b2bd56da47">print</a>( <span class="stringliteral">&quot;total error: {}&quot;</span>.format(mean_error/len(objpoints)) )</div></div><!-- fragment --> <h2>Additional Resources </h2>
<h2>Exercises </h2>
<ol type="1">
<li>Try camera calibration with circular grid. </li>
</ol>
</div></div><!-- contents -->
<!-- HTML footer for doxygen 1.8.6-->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Apr 2 2021 11:36:37 for OpenCV by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
</a> 1.8.13
</small></address>
<script type="text/javascript">
//<![CDATA[
addTutorialsButtons();
//]]>
</script>
</body>
</html>
